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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 
1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on 16 September 2004 has been entered. 



Claim Objections 

2. The following sections of 37 CFR §1.75(a) and (d)(1) are the basis of the following 
objection: 



(a) The specification must conclude with a claim particularly pointing out 
and distinctly claiming the subject matter which the applicant regards as his 
invention or discovery. 

(d)(1) The claim or claims must conform to the invention as set forth in 
the remainder of the specification and the terms and phrases used in the 
claims must find clear support or antecedent basis in the description so 
that the meaning of the terms in the claims may be ascertainable by 
reference to the description. 



3. Claims 3-6, 30, and 34 are objected to under 37 CFR § 1 .75(a) and (d)(1) as failing to 
particularly point out and distinctly claim the subject matter that the applicant regards as the 
invention. 

Regarding claims 30 and 34, the claims recite "a foreground area" and then improperly 
refer to "the foreground areas ." Correction is required. Claims 3-6 depend from claim 30 and are 
objected to for the same reason. 
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Claim Rejections - 35 USC § 103 

4. The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 

5. Claims 30 and 34 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 5,644,765 by Shimura in view of U.S. Patent 6,584,221 by Moghaddam et al. 
("Moghaddam") and U.S. Patent 5,805,215 by Mizoguchi. 

Regarding claims 30 and 34, Shimura discloses a method/computer program (figure 4) 
for detecting duplicate (i.e. substantially similar) images comprising the steps of: 

providing at least two images captured at determinable times from original scenes (figure 
2: image 20 and database of images 33 are provided; figure 2, elements 43, 12, and 32 and 
column 4, lines 1-9: additional information, such as date and time of registration, is associated 
with each of the images and denote determinable times of capture); 

computing an indication of image content for each image by dividing each image into 
blocks, computing an indication of image content in each block (e.g. number of black pixels in 
each block comprises a "feature" that indicates the image content; column 3, lines 33-37), and 
comparing the computed indication of image content in each corresponding block for the two 
images to generate a similarity metric for each block (32, figure 5 and 52, figure 2: the features 
of corresponding blocks (i.e. the computed indications) of the images are compared to generate 
similarity metric); 

determining the time of original capture of each of the images (figure 4, SI 3: additional 
information (e.g. time of registration) is determined for each image); and 
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evaluating the similarity metrics and the time of original capture (figure 4, S16 and SI 8) 
to determine whether the images are duplicate images (i.e. whether the images are substantially 
similar with regards to the content of the image blocks and the time of registration); 

Shimura computes the indication of image content in each block and generates a 
similarity metric for the entire image by comparing the indications (features) for each block in 
the image to those stored in the database. 

However, Shimura does not appear to disclose assigning two or more predetermined 
blocks to represent a foreground area, and then computing an indication of image content in the 
foreground area to generate a similarity metric for the foreground area. 

Moghaddam discloses a method for retrieving an image from a database by comparing 
histograms of an input image to histograms of images stored in a database. In particular, 
Moghaddam discloses partitioning an input image into a plurality of blocks and computing an 
indication of image content (i.e. an histogram) for each block (see figure 2). Then, a user is 
allowed to select a region of interest comprising a number of predetermined blocks to represent 
an object of interest (i.e. a foreground area) (see figure 3). The histogram of the region of interest 
is computed and matched to the histograms of database images for similarity. Column 3, line 53 
through column 4, line 44. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura by Moghaddam to compute the indication of image content for two or more 
predetermined blocks representing a foreground area of interest and to generate a similarity 
metric for the foreground area, as claimed, since Moghaddam teaches that allowing a user to 
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select a foreground area of interest to be matched to a database provides the user with "a more 
powerful retrieval tool," wherein the user is able to specify a particular region of high interest to 
be retrieved rather then simply attempting to retrieve an entire image. Column 1, lines 58-67. 

Shimura teaches inputting the images using a scanner 1 1 (or the like), figure 1 . 

Shimura is silent to using a photographic camera to capture the images. 

Mizoguchi discloses a photographic camera having the ability to store additional time 
data along with the images captured by the camera (1, figure 1). The time/date of capture is 
encoded in memory 51 (figure 10) with each image provided by the camera (column 6, lines 57- 
65). The date/time information is then used as criteria for retrieving images (column 8, lines 7- 
28). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura and Moghaddam by Mizoguchi to utilize Mizoguchi' s photographic camera 
to input images and the time of capture of the images, since Mizoguchi 's camera functions as an 
image input device in substantially the same fashion (i.e. to provide an electronic representation 
of an object) as Shimura's image input device 11, and Mizoguchi's camera eliminates the need 
for the user to manually input the time of capture of the image. 

6. Claims 3-6 and 20-23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 5,644,765 by Shimura et al. ("Shimura") in view of U.S. Patent 6,584,221 by 
Moghaddam et al. ("Moghaddam") and U.S. Patent 5,805,215 by Mizoguchi, as applied to claim 
30, and further in view of U.S. Patent 6,400,853 by Shiiyama. 
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Regarding claims 3, 4, 20, and 21, neither Shimura nor Moghaddam expressly disclose 
dividing the images into 4x4 or 3x3 blocks. However, at the time the invention was made, 
dividing an image into a small number of blocks and processing each block was common in the 
art and would have been an obvious modification to those skilled in the art. 

As Shiiyama shows (figure 5), it is conventional to divide an image into 3x3 blocks for 
the purposes of determining the similarity between images. 

Regarding claims 5 and 22, Shimura and Moghaddam disclose the indication of image 
content in each block comprises computing an histogram for each block. Shimura computes a 
simple histogram as the number of black pixels (column 3, lines 34-37), and Moghaddam 
computes a joint histogram of each block (see column 3, lines 64-65). 

Regarding claims 6 and 23, Shimura and Moghaddam disclose the similarity metrics are 
histogram intersection metrics - Shimura compares simple histograms, comprising the number of 
black pixels, for corresponding blocks to determine the intersection, or similarity, of the blocks, 
and Moghaddam uses conventional histogram intersection metrics to determine the similarity 
(see column 4, lines 45-60). Shimura also discloses using the time difference between capture of 
the two images to determine whether the images are duplicate images. Figure 4, SI 3: time 
information of the two images is compared; figure 4, SI 7: features of the two images are 
compared. Both criteria are used to determine the similarity of the images. 

7. Claims 27, 31, and 33 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 5,644,765 by Shimura in view of U.S. Patent 6,584,221 by Moghaddam et al. 
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("Moghaddam"), "Indexing Via Color Histograms" by Swain et al. ("Swain"), and U.S. Patent 
5,805,215 by Mizoguchi. 

Regarding claims 31 and 33, Shimura discloses a method (figure 4) for detecting 
duplicate (i.e. substantially similar) images comprising the steps of: 

providing at least two images captured at determinable times from original scenes (figure 
2: image 20 and database of images 33 are provided; figure 2, elements 43, 12, and 32 and 
column 4, lines 1-9: additional information, such as date and time of registration, is associated 
with each of the images and denote determinable times of capture); 

computing an indication of image content for each image by dividing each image into 
blocks, computing an indication of image content in each block (e.g. number of black pixels in 
each block comprises a "feature" that indicates the image content; column 3, lines 33-37), and 
comparing the computed indication of image content in each corresponding block for the two 
images to generate a similarity metric for each block (32, figure 5 and 52, figure 2: the features 
of corresponding blocks (i.e. the computed indications) of the images are compared to generate 
similarity metric); 

determining the time of original capture of each of the images (figure 4, SI 3: additional 
information (e.g. time of registration) is determined for each image); and 

evaluating the similarity metrics and the time of original capture (figure 4, S16 and SI 8) 
to determine whether the images are duplicate images (i.e. whether the images are substantially 
similar with regards to the content of the image blocks and the time of registration); 

wherein the step of evaluating the similarity metric for each block and the time of capture 
comprises comparing one or more blocks of one image, using an histogram intersection metric 
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to corresponding blocks of another image and using the difference between capture of the two 
images to determine whether the images are duplicate images (i.e. Shimura utilizes the additional 
information indicating the time of capture and the intersection, or comparison, of features of 
corresponding blocks of pixels - the features comprising the number of black pixels (i.e. an 
histogram of black pixels) - to determine whether tow images are duplicates). 

Shimura does not disclose utilizing the claimed equation to compute the similarity of 
image blocks. 

Moghaddam discloses a method for retrieving an image from a database by comparing 
histograms of an input image to histograms of images stored in a database. In particular, 
Moghaddam discloses partitioning an input image into a plurality of blocks and computing an 
indication of image content (i.e. an histogram) for each block (see figure 2). The histograms of 
the blocks are matched to the histograms of database images for similarity. Moghaddam utilizes 
conventional histogram intersection metrics, such as that of Swain et al., for comparing the 
histograms of corresponding blocks. Column 3, line 53 through column 4, line 44. 

Swain discloses the conventional method of histogram intersection utilized by 
Moghaddam. Swain's histogram intersection metric H(I t M) in §2.1 on page 391 is the same as 
the claimed metric. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura by Moghaddam and Swain to utilize the claimed histogram intersection 
metric since Moghaddam discloses that, for the purposes of determining the similarity of image 
blocks on the basis of histograms, it is both preferred and conventional to utilize Swain's metric. 
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Shimura teaches inputting the images using a scanner 1 1 (or the like), figure 1. 

Shimura is silent to using a photographic camera to capture the images. 

Mizoguchi discloses a photographic camera having the ability to store additional time 
data along with the images captured by the camera (1, figure 1). The time/date of capture is 
encoded in memory 51 (figure 10) with each image provided by the camera (column 6, lines 57- 
65). The date/time information is then used as criteria for retrieving images (column 8, lines 7- 
28). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura and Moghaddam by Mizoguchi to utilize Mizoguchi 5 s photographic camera 
to input images and the time of capture of the images, since Mizoguchi 5 s camera functions as an 
image input device in substantially the same fashion (i.e. to provide an electronic representation 
of an object) as Shimura 5 s image input device 11, and Mizoguchi 's camera eliminates the need 
for the user to manually input the time of capture of the image. 

Regarding claim 27, Shimura teaches inputting additional information (12 and 43, figure 
1) of input images, such as the date of capture, using a keyboard, but is silent to determining the 
time of capture by extracting encoded time information from images provided by a digital 
camera. That is, Shimura does not disclose how the time information is acquired, only that it is 
input. 

Mizoguchi discloses storing additional time data with images captured using a digital 
camera (1, figure 1). The time/date of capture is encoded in memory 51 (figure 10) with each 
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image provided by the camera (column 6, lines 57-65). The date/time information is then used as 
criteria for retrieving images (column 8, lines 7-28). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura by Mizoguchi to extract time information from images provided by a digital 
camera and input the extracted time information using Shimura' s additional-information input 
units (43 and 12, figure 12) since Mizoguchi teaches that information pertaining to the time of 
capture is stored with the image at the time of capture, and said time information is extracted, or 
otherwise accessed, for determining the time of capture. 

8. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent 
5,644,765 by Shimura in view of U.S. Patent 6,584,221 by Moghaddam et al. ("Moghaddam"), 
"Indexing Via Color Histograms" by Swain et al. ("Swain"), and U.S. Patent 5,805,215 by 
Mizoguchi, as applied to claim 33, and further in view of U.S. Patent 4,143,956 by Miyagawa. 
7. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over Shimura in view 
of Mizoguchi and U.S. Patent U.S. Patent 4,143,956 by Miyagawa. 

Regarding claim 26, Shimura teaches inputting additional information (12 and 43, figure 
1) of input images, such as the date of capture, using a keyboard, but is silent to determining the 
time of capture by extracting encoded time information from a film strip used to capture the 
images. That is, Shimura does not disclose how the time information is acquired, only that it is 
input. 
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Miyagawa discloses storing additional time data onto a film strip used to capture an 
image (column 1, lines 34-39). The time data indicates the time/date at which the image was 
captured by the camera. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Shimura and Mizoguchi by Miyagawa to extract time information from a film strip 
used to capture an image and input the extracted time information using Shimura' s additional- 
information input units (43 and 12, figure 12) since Miyagawa teaches that information 
pertaining to the time of capture is stored on the film strip at the time of capture, and said time 
information is extracted, or otherwise accessed, for determining the time of capture. 

9. Claim 28 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent 
5,644,765 by Shimura in view of U.S. Patent 6,584,221 by Moghaddam et al. ("Moghaddam"), 
"Indexing Via Color Histograms" by Swain et al. ("Swain"), and U.S. Patent 5,805,215 by 
Mizoguchi, as applied to claim 33, and further in view of U.S. Patent 6,163,622 by Abdel- 
Mottaleb et al. ("Abdul-Mottaleb"). 

Regarding claim 28, Shimura discloses evaluating the similarity metric and the time of 
capture to determine whether the images are duplicates but is silent to the steps of generating an 
average of the similarity metrics for the blocks and evaluating the average of the similarity 
metrics as claimed. 

Abdel-Mottaleb discloses a system that determines the similarity between images. In a 
method similar to that of Shimura, Abdel-Mottaleb, divides the images into blocks and computes 
an indication of image content for each block (figure 3). Then, for the purposes of determining 
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the similarity between two images, Abdel-Mottaleb discloses generating an average of the 
similarity metrics for the blocks (equation (8), column 7). The average of the similarity metrics 
S a k is then used for evaluating the similarity between the images. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to generate and evaluate the average of similarity metrics for the blocks as claimed since Abdel- 
Mottaleb teaches that using an average (i.e. the median) of block similarities avoids the problem 
of too much emphasis being placed on any one of the block similarities (column 7, lines 38-50). 



Allowable Subject Matter 

10. Claims 8-17 are allowable. 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

U.S. Patent 6,721,449 by Krishnamachari 
U.S. Patent 5,796,428 by Matsumoto et al. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (703) 306-3489. 
The examiner can normally be reached Monday through Thursday from 8:00 to 5:30. The 
examiner can also be reached on alternate Fridays. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au, can be reached on (703) 308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the TC 2600 Customer Service Office whose telephone number is (703) 
306-0377. 



Group Art Unit 2623 
16 February 2005 
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